Search for: All records

Incorporating recordings of teacher-student conversations into the training of LLMs has the potential to improve AI tools. Although AI developers are encouraged to put "humans in the loop" of their AI safety protocols, educators do not typically drive the data collection or design and development processes underpinning new technologies. To gather insight into privacy concerns, the adequacy of safety procedures, and potential benefits of recording and aggregating data at scale to inform more intelligent tutors, we interviewed a pilot sample of teachers and administrators using a scenario-based, semi-structured interview protocol. Our preliminary findings reveal three "paradoxes" for the field to resolve to promote safe, fair, and trustworthy AI. We conclude with recommendations for education stakeholders to reconcile these paradoxes and advance the science of learning. 

Improving equity in K-12 Computer Science (CS) education benefits from the collaboration of classroom teachers, school counselors, and school leaders. This paper presents the outcomes of a pilot program that brought together cross-functional teams consisting of CS teachers, school counselors, and administrators. Over the course of a year, these teams attended monthly, equity-focused workshops, leveraging pre-existing materials from affordable, high-quality, research based programs. The use of these resources demonstrated benefits of sequencing and synthesizing existing programs. Evidence from surveys and interviews shows that the workshops promoted learning and fostered collaboration between the cross-functional teams that would not have happened otherwise. Participants were motivated by the program, and they generated ideas that turned into actionable projects to promote CS education equity in their schools. While the initiative was well received, areas for improvement were identified, particularly, in school recruitment, workshop structure, and evaluation. This pilot initiative demonstrates that equity-centered programs comprised of cross-functional teams can help achieve systemic improvement of CS education equity 

Implementing high-quality professional learning on diversity, equity, and inclusion (DEI) issues is a massive scaling challenge. Integrating dynamic support using natural language processing (NLP) into equity teaching simulations may allow for more responsive, personalized training in this field. In this study, we trained machine learning models on participants’ text responses in an equity teaching simulation (494 users; 988 responses) to detect certain text features related to equity. We then integrated these models into the simulation to provide dynamic supports to users during the simulation. In a pilot study (N = 13), we found users largely thought the feedback was accurate and incorporated the feedback in subsequent simulation responses. Future work will explore replicating these results with larger and more representative samples 

Role-plays of interpersonal interactions are essential to learning across professions, but effective simulations are difficult to create in typical learning management systems. To empower educators and researchers to advance simulation-based pedagogy, we have developed the Digital Clinical Simulation Suite (DCSS, pronounced "decks"), an open-source platform for rehearsing for improvisational interactions. Participants are immersed in vignettes of professional practice through video, images, and text, and they are called upon to improvisationally make difficult decisions through recorded audio and text. Tailored data displays support participant reflection, instructional facilitation, and educational research. DCSS is based on six design principles: 1) Community Adaptation, 2) Masked Technical Complexity, 3) Authenticity of Task, 4) Improvisational Voice, 5) Data Access through "5Rs", and 6) Extensible AI Coaching. These six principles mean that any educator should be able to create a scenario that learners should engage in authentic professional challenges using ordinary computing devices, and learners and educators should have access to data for reflection, facilitation, and development of AI tools for real-time feedback. In this paper, we describe the architecture of DCSS and illustrate its use and efficacy in cases from online courses, colleges of education, and K-12 schools. 

In this study, we examine the outcome of a four-day workshop with 24 Teacher Educators (fellows) who were supported in using two tools - Teacher Moments (TM) and Eliciting Learner Knowledge (ELK). The tools are designed for authoring, implementing, and research Digital Clinical Simulations in education. The simulations centered around issues of equity in K-12 computer science education to provide in-/pre-service teachers with opportunities to practice high-stakes interactions in low-stakes settings. We operationalize the technology adoption of the fellows through the notions of self-efficacy, help-seeking, and technology concerns to recognize the potential barriers they faced in transitioning from authoring to implementing and research design. Finally, we note the fellows' implementation plans in the ensuing academic year and examine potential collaborations amongst them using social network analysis. Our results reveal how a small group of fellows, spanning major regions of the U.S., generate a broad range of scenarios, as well as clusters of scenarios, enabling simulation-based research supported by collaboration. 

Online education is rapidly expanding in response to rising demand for higher and continuing education, but many online students struggle to achieve their educational goals. Several behavioral science interventions have shown promise in raising student persistence and completion rates in a handful of courses, but evidence of their effectiveness across diverse educational contexts is limited. In this study, we test a set of established interventions over 2.5 y, with one-quarter million students, from nearly every country, across 247 online courses offered by Harvard, the Massachusetts Institute of Technology, and Stanford. We hypothesized that the interventions would produce medium-to-large effects as in prior studies, but this is not supported by our results. Instead, using an iterative scientific process of cyclically preregistering new hypotheses in between waves of data collection, we identified individual, contextual, and temporal conditions under which the interventions benefit students. Self-regulation interventions raised student engagement in the first few weeks but not final completion rates. Value-relevance interventions raised completion rates in developing countries to close the global achievement gap, but only in courses with a global gap. We found minimal evidence that state-of-the-art machine learning methods can forecast the occurrence of a global gap or learn effective individualized intervention policies. Scaling behavioral science interventions across various online learning contexts can reduce their average effectiveness by an order-of-magnitude. However, iterative scientific investigations can uncover what works where for whom.